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changing perturbations 
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Michigan State University and the University of Michigan 

Abstract: Non-linear renewal theory is extended to include random walks 
perturbed by both a slowly changing sequence and a stationary one. Main 
results include a version of the Key Renewal Theorem, a derivation of the 
limiting distribution of the excess over a boundary, and an expansion for the 
expected first passage time. The formulation is motivated by problems in se- 
quential analysis with staggered entry, where subjects enter a study at random 
times. 



1. Introduction 

Non-linear renewal theory concerns sequences of the form Z° = SVi + Cm where S n is 
a random walk with a finite positive drift, p say, and is a sequence that changes 
slowly, as in ([5]) and ([6]) below. The main results are extensions of the Renewal 
Theorem and selected corollaries from the random walk S n to the perturbed random 
walk Z°. For example, letting t° denote the first passage time t° = inf{n > 1 : Z° > 
a}, it is shown that E(t°) = (a + p— rf) / p + o{\) under natural conditions, where p is 
the mean of the asymptotic distribution of the excess Z*o — a and r\ = linin^oo _E(£„). 
The formulation is due to Lai and Siegmund [B[ and (fl. The early development of 
non-linear renewal theory is described in Woodroofe [8|, and more general results 
have since been obtained by Zhang The formulation is motivated by problems 
from sequential analysis, where the stopping times of many sequential procedures 
may be written in the form t°. Applications to sequential analysis are described in 
Q and in Siegmund's book 

Sequential problems with staggered entry (where subjects arrive according to a 
Poisson process, say) lead to random walks perturbed by both a slowly changing 
term and a stationary one, which is not slowly changing. Examples are described 
below. In such cases, there is interest in processes of the form 

(1) z n = s n + i n + Cm 

where S n and £ n are as above and £ n is a stationary sequence and in the associated 
first passage times, 

(2) t a = inf{n > n : Z n > a}, 
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where no is a fixed (throughout) positive integer. Kim and Woodroofe [4j considered 
a simple case of ([1]) in which £„ = 0, obtaining versions of the Renewal Theorem 
and its corollaries in that context. They also applied their results to the problem 
of testing a simple hypothesis about the mean of an exponential distribution, but 
applications were severely limited by the absence of a slowly changing term. The 
purpose of this paper is to supply a non- linear renewal theory in the context of ([T]) . 

For example, consider a model in which patients arrive according to a Poisson 
process, with rate A > say, are treated, and then live for an exponentially dis- 
tributed residual life with unknown failure rate 9 > 0, (or are cured after an 
exponentially distributed time with unknown success rate). Letting Tk-i denote 
the arrival time of the fc th patient with To = 0, the data available at time r„ are 
L n ,k = min[L fc ,T„ - r fe _i] and 5 n , k = l{L k < r„ - r fc _i}, k = 1, . . . ,n, and the 
log-likelihood function is 

l n (6) = K n log(d) - 6T*, 

where K n = <5„.i + • • • + 5 n>n and T* = L n ,\ + • • • + L n _ n . Here K n counts the 
number of deaths (or cures), and T* is called the total time on test statistic. To see 
how staggered entry leads to processes of the form ([1]), write 

n n 

T n = ^ L k - ^[£„_ fe+ i - (r„ - r n _ fc )] + . 
fe=i fe=i 

The first term on the right is a random walk, and the second may be written as 

n k—1 oo fc— 1 

^[L„_ fc+ 1 - Vn-j)]+ = ^[£„-fc+l - Vn-j)]+ + 0(1) W.p.l, 

fe=l j=0 k=l j=0 

where r/k = Tk — Tk—\. If the (sequences of) arrival times and lifetimes are indepen- 
dent, then the last sum is a stationary sequence and the o(l) is slowly changing. So, 
T* is of the form JT]). Combining these observations with Taylor series expansions, 
leads to processes of the form (Q} for many other statistics. Specific examples are 
considered in Section 4. 

The main results are stated and proved in Section [3] Section [2] contains some 
preliminary lemmas. 

2. Preliminaries 

In Sections 2 and 3, . . . W-x, Wq, W\, W2, ■ ■ ■ denote i.i.d. random elements with 
values in a Polish space, W say, Xk — <p(Wk) and = ip{Wk)-, where <p : W — > K 
and if '■ VV — > K d are Borel measurable functions, S n = X\ + • • • + X n , and 
T n = Y% + • • • + Y n . Suppose throughout that X\ has a non-arithmetic distribution 
with a finite, positive mean /1 and that E(Yk) = 0. Next, let £ n be a sequence of 
the form 

(3) e« = «W n ,W r „_i J ...), 

where £ is measurable on W N and N = {0, 1,2,.. .}. Let C« = Cn(Wi, . . . , W n ) be 
random variables of the form 



(4) 



Cn — Cn + Cm 
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Cri T QT n , 
n 



where 
(5) 

Q is a symmetric d x d matrix, and 

(6) limn" J2 P (\Ck\>^) 



o 



n < fc < n + n 3 

for each e > for appropriate k > 0. Finally, suppose (throughout) that 
(7) E\X^ + E\^ + EWY^y + sup E\( n \ p < < oo, 

n>n ( ) 

for appropriate p^, p y , p^, and > 2. Then 



(8) 



Cn C = XI 



where Xi i> • • • >Xi d are independent chi-square random variables and Ai, . . . , Xd are 
constants. Let L denote the distribution function of C- Then L is continuous unless 
Y[QY\ = w.p.l. To avoid trivialities we suppose that P[Y{QYi ^ 0] > for the 
remainder of the paper. 

The conditions imposed above with k = and p x = p y = p^ = = 2 are 
Standing Assumptions and are not repeated in the statements of results. Additional 
moment conditions are imposed by requiring ([7]) for higher values of p x , p y , p^ , 
and p<j, and ^ for selected k > 0. 

With Z n = S n + £» + Cn, as in U), and 1/3 < g < 1/2, let 



(9) 



m = m a 



1 - a" 9 



(10) 



M = M n 



l + a~ q 



(11) 
and 
(12) 



A (q,a) = ^P(Z„ > a), 



Ai(?,a,6)= ^ P(Z n <a + b) 

n=M+l 



where [x\ is the greatest integer that is less than or equal to x. Below, there is 
special interest in q's that are slightly larger than 1/3. 

Lemmas [1] and [5] are extensions and improvements of Lemmas 1 and 4 in Q ■ 
The proof of Lemma 2 is virtually identical to the proof of Lemma 4 of [4] . It is 
only necessary to replace the W„ of 4] by W * . The proof of Lemma 1 is outlined 
in the Appendix. 
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Lemma 1. If p x > (2 — q)/(l — q), then 

(13) Urn {A {q,a) + A 1 (q,a,-a 1 - q )+ ^ P[t Q > n]} = 0. 

Ifp x > 2/(1 - q), p e Ap ( >(2- q)/{\ - q) then 

(14) lim \aP[t a < to] + aP[t a > M} \ = 0. 

For Lemmas 2 and 4, let B denote the Borel sets of W, B k = B® ■ ■ -®B (k 
factors), B n the product sigma-algebra in W N , and B 1 the product sigma- algebra 
in W z , where Z = {. . . - 1, 0, 1, . . .}. Further, let 

W„ = (...W n ^,W n ), 
W* n = (...W n - 1 ,W n ,W n+1 ,...), 

n 

I^m 7 n — ^ ^fc; 
k— n— m+1 

Cm.n ^m.nQ-^m,'" 

m 

and 

Thus, the j component of W* is W n +j. Observe that £ n + ( m ,n is a stationary 
process (in n) for fixed a. So, .Zm,n is a random walk perturbed only by a stationary 
sequence (that depends on a). 

Lemma 2. Given < a < 1, let fi — (1 — a 2 ) /a. Then there are constants 
Ki = K\ a and K 2 = K 2a for which 

oo 

P[Wn e B, a<S n <a + b]< Kx{\ + b)P[W* e B] a 

n=Q 

and 

V P[W; e P, a < Z m , n < a + b] < K 2 (l + b)\ (1 + |£ + Cm,o|)^P 

/or aZ/ B eB z and < a, 6 < oo. 

Lemma 3. Let to, M 6e as m (0) and (fit))) and e > 0. Then 

M 

(15) lim V P[|C„ - Cm,n| > e] = 0; 

a — >oo * — » 

n— m+1 

and if n — 1 in ([6]) and p y > 1/q in ([7J) ; iften 

(16) lim aP[ max \( n - (L n | > e] = 0. 

a-too m<n<M 
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Proof. By ([6]) , it suffices to establish (|T5|) with £„ replaced by C,' n . Towards this end 
observe that 

Cri Cm.n I ) T QT n H (T QT n T QT m n) 

\n to ) to 

and 

P{\C-Cm,n\>e)<I n + U n 

where 

In = P 



1 1 

n to 



T' n QTn\ > \ 



and 



Hn — P I \T n QT n T QT m n \ > 

'to 2 



Since have finite fourth moments, there is a constant C for which _E[(T^QT„) 2 ] < 
Cn 2 and, therefore 

T < P f\T'HT I > ^ < 4 ( M - TO ) 2 ^ 2 . 8C(M - TO) 2 

In < P \T n QT n \ > — — — — < Cn < T — 



2(M-m) 

for all m < n < M and all sufficiently large a. For //„, first observe that T n = 
T m ,n + T n -m where T m , n and T„_ m are independent, and that T m) „ has the same 
distribution as T m . So, 

Hn < P[Wm,n\\ WQW W T n-m\\ > ~] + P[\\Q\\ \\T n - m \\ 2 > ™] 

C'm 2 (n — m) 2 C'(n-m) 2 
< - — | i '— 



„ 2C (M - to) 2 
— e 4 to 2 

for < e < 1, where C is a constant depending on the 4th moment of ||Yi|| and 
||Q||. Since <7> 1/3, 

4^ [T TT ^ 8C + 2C'{M-m) 3 n 
e 4 to 

n— m+1 

establishing (fT7)j) . 

If ([6]) holds with k = 1, then P[max m <„<M |C"I — e ] = °(V a ) f° r eacn e > by 
Boole's Inequality, and it suffices to establish (fTI))) with Cn replaced by £„■ For this 

P[ max |C -Cm.nl >6] <I* a +K 

m<n<M 



where 



P a =P[ max mQT n \> 



m 2 e 



L m<»<A/' ' ~ 2(M- TO) J 

and 

7726 7776 

P a * - P[ max 2|T^ n QT„_ m | > — ] + P[ max \T' n _ m QT n _ m \ > —). 

7n<n<M 4 m<n<M 4 
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By the Submartingale Inequality, there is a constant C" for which 

I! <C"\ 2{M ~ m) ] Pv MPy, 

771 € J 

which is o(l/ a) if p y > 1/q. For II*, observe that T m ^ n = ^fcln-m+i Y k + 
Sfe=M-m+i ^k- So, using the Submartingale Inequality again and possibly enlarg- 
ing C" and 

II* < C" \—] 2pv MP* (M - mfy + C" \— 1 Pv (M - mfy , 
which is again o(l/a) if p y > 1/q. □ 

Let Cfc the collection of cylinder subsets in W N with base in W k , and write 
members of C k as W N x C, where C £ W k . Also let C = U^ =1 C k be the class of 
cylinder sets. 

Lemma 4. If p x > (2 — g)/(l — q), f/ien /or eac/i < 6, z < 00, 



n=m+l 



lim I V P[W„ S B, Cn < z, a < S„ <a + b 
b 



P[W £B,Cm, n <z}\ = 



uniformly with respect to B £ C m ; and 



00 , 
(18) lim V P[W n e B,(n < z,a < S n < a + b] = -P[W £ B]L(z) 

a— >oo * ^ li 

n.—l 

for each B £ B N . 

Proof. First consider (|17p when z — 00 and let V denote the renewal function for 
Si, S 2 , ■ ■ ■ ■ If B £ C m , so that B = W N xC for some C £ # m , then 

00 . 

V P(W„ €B, a<S n <a + b)= V(a- S m , a - S m + b)dP, 

n=m+l JW m eB 

by independence and symmetry, as in Lemma 2 of As o -> 00, a - S m — 
(a — m/i) + (m/i — S m ) — ► 00 in probability since q < 1/2 and, therefore, V(a — 
S m , a — S m + b) — > b/ fi in probability, by the Renewal Theorem for random walks, 
[|, Ch. 12. It follows that 

I / V(a-S m ,a- S m + b)alP - -P[W m £ B] \ 

JW m £B M 

< / \V(a -S m ,a- S m + b)--\a!P^0 

uniformly with respect to B £ C m . That is, (|17p holds when z — 00, since P[W m £ 
B]=P[W €B}. 
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Now consider the case z < oo. Using Lemmas Q] and [3l 



n—m+l 



J2 P\W n £ B,( m , n <z-e, a<S n <a + b\- o(l) 

OO 

< P t W ™ e B ' C» ^ z ' a < S„ < a + &] 

n— m+1 
oo 

< P t W « e B ' Cm.n < z + e, a< S n <a + b] + o(l). 

n=m+l 

for any e > 0, where the o(l) terms are independent of P £ £? N . Let 

B a ± = {weB:Cm,o(w)<z±e}. 

Then the summands on the first and last lines are P[W n £ B^, a < S n < a + b], 
and P[W £ B±] = P[W £ B, Cm,o < z ± e]. If B e C m , then also B± £ C m and, 
therefore, 

oo , 

lim { V P[W„ £ B±, a < S n < a + b] - -P[W £ B, ( m , < z ± e}\ = 0, 

a— too L * — ' n ) 

n— m+1 

uniformly with respect to B E C m . It follows that 

lim sup | P t W » G B ' Cm.n < z, a < S„ < a + 6] - -P[W G P, Cm.O < *]| 

a — >oo , i M 

n— m+1 

< limsupPfz — e < Cm,o < z + e ] = £( z + e ) — £( z — e ) 



uniformly with respect to B £ C m and this establishes (|T7|) . since e > was arbi- 
trary. 

Now consider (|18[) . For any B £ £? N , and e > 0, there is a cylinder set Bq £ C for 
which P[Wq £ PAPq] < e, where A denotes symmetric difference, in which case 



| ]T P[W„ G P, Cm ( o < z, a < S n < a + b] 
n=l 

- P[W„ G B , Cm,o < z, a < S n < a + b}\ < K x {\ + b)yfe, 
by Lemma [21 Combining this inequality with (|17p, 

oo 

limsup | P[W„ G P, Cm,™ < z, a < S n < a + b] 

a — »oc .. 

n—1 

- -P[W G Bo, Cm,0 <Z]\< K X (l + b)y/l 

A* 

If P G Cfc, write T m ,o = (Y- m +i H hF_fc_i) +Tfe+i,o- Here Po and the first term 

are independent and the second is bounded as m — > oo. It follows easily that 

lim P[W G P , Cm.o <z] = P[W G B ]L(z), 

a — >oo 

from which the Theorem follows by letting e — > 0. □ 
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3. Main results 

There are four main results: a renewal theorem, uniform integrability, the limiting 
distribution of the excess over the boundary, and properties of the first passage 
time t a . For technical reasons, they are presented in that order. Recall that 1/3 < 
q < 1/2. 

Theorem 1. If p x > (2 - q)/(l - q), then 

(19) lim V P[W n e B, <y,a<Z n <a + b] = -P[W £ B]L(y) 

a~ »oc * — ' // 

n— 1 

/or eac/i J5 € and < y < oo . 

Proof. It suffices to establish the theorem for < b < /i. Let 

oo 

Ja(x,V,z) = ^P[W n X,(n < y,a < S n < a + z] 

n=l 

and 

oo 

y,a < Z n < a + z] 

n=l 

for < a, b < oo and y, z € K. Then 

(20) lim J a {x,y,z) = -P[W G B,£ < := Joo(x,y,z) 

a — >oo ^£ 

by Lemma[4]with the -B in Lemma [4] replaced by BD{^o < x}. Next, let T(x, y, z) — 
(x, y, x + y + z), where addition is understood modulo \x. Then T is continuous a.e. 
(Joo) and _ftT a = J a oT^ 1 , and Joo ° = Joo. Theorem [1] now follows from the 
Continuous Mapping Theorem. □ 

The uniform integrability that is needed will be deduced as corollaries to Theo- 
rem [2] below. In its statement, A a denotes the event 

A a = {m<t<M, max |C« - Cm.«| < !}■ 

m<n<M 

In the proofs U n = Z n - Z n -i = X n + (£ n - + (£„ - ( n -i) and U m ,n = 

Zm,n Zjyi^ji — X — X n -\- (£ n ^n— l) ~t~ (Cm,n Cm,7i— l)- 

Theorem 2. Suppose p x > (2 — — q). If < a < 1, £/iere is a constant 

K3 = Kj, M for which 

P[A a ,W* ta eB]<K 3 \ f (1 + |£o + Cm,o\f (1 + L> m ,o) 1+/3 rfP 

/or aZZ Borel sets B G £> N , where (3 = (1 — a 2 )/a. 

Proof. Since i a = n implies Z„_i < a < Z„, 
a/ 

P[A a , W t * a e B] < P i W n e B, ^m,„-i < a + 1, Z m ,„ > a - 1] 

M 

< J2 P[WleB,k-KZ m , n _ 1 <k,U m , n >a-k-l] 

n=m+l k< LaJ+2 

<2K 2 V [/ (l + leo + Cm.oD^P 

fe<Laj+2 L JWo*eB,t/ m ,o>a-fc-l 
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using Lemma [2] in the last step. Letting j — \_a\ — k — 1, the latter sum is at most 



8K 2 



(l + lCo + Uol^dP 



W *6B 



, =0 L Av " 



es,£/ m , >j 



(i + ieo+Cm,oi)^p 



of which the sum is at most 



1 x a/3 



3=0 

< 

< 



+ 3' 



(l + leO + Cm.oD^l + ^o)^ 



Wo'eB,a m ,o>3 



OO „ 1 OO „ 

[E(i^)*r-[EjC. 

j=0 J j=0 Jw o 



(l+|e0+Cm,0|) /3 (l + t/m,0)' 3 dP 



, 1+a 



3=0 



IWo*eB,U mi0 >3 



Wo*eB 



The theorem follows easily. □ 

The uniform integrability over A a of R a , £,t a + Ct„ is easy consequence. 
Corollary 1 (Under the conditions of Theorem^. R a lA a are uniformly integr able. 
Proof. If A a occurs, then R a < XJ ta < Umt a + 2, and 



P[A a ,U m . ta >r}<K 3 
K3 

K 3 



(l + \£o + Uo\) f3 (l + U m nf +1 dP 



U m . >r 



< 



U m .o>r 



< 



r Q7 



(l + \Zo + L,o\) 2 ^ +1 dP 



U m .o>r 



a0 
2p + y + l 



U m .o>r 



(l + U+.of^'dP 



2/3 + t + I 



for any < a < 1 and 7 > 0, where /3 = (1 — a 2 )/a and are as in Theorem[2l 
The conditions of the theorem require p x > 5/2. So, there are a < 1 and 7 > 1/a 
for which 2/3 + 7+1 < p^. Then the last two integrals are bounded in a and r, and 
the uniform integrability of U m j a over A a follows. □ 



Corollary 2. |£t a + Ctjl^4 a are uniformly integrable. 
Proof. Similar to that of Corollary [T] 



□ 



The next theorem uses the following easily verified lemma: If V a and V are 
random vectors for which (the distributions of) V a are tight and 

(21) limsupP[V a eK}< P[V G K) 

a—>oo 

for all compact rectangles then V a converges in distribution to V 



Non-linear Renewal Theorem 



185 



Theorem 3. If p x > (2 — q)/(l — q) then for all < r < s < oo, 

lira P[r < R a < s, £ ta <y,(t a <z} = - I P[ inf Z* > u, Co < t/]du x 

a^oo |U J,, i<-l 

w/iere Z* = + h X + Co — £7 • 

Proof. If 7 and J are intervals, then 

P[r < i?a < s,Ct„ G I, Ct a G J] 

OO 

= P[* a > n, Cn G I, Cn G J, a + r < Z n < a + s] 

71=1 

Next, for any e > and any integer k, 

P[t a >n, Cn G I, Cn G J, a + r < Z n < a + s] 

< P[min(Z J 1 l - •) > r - e, Cn € I, Cn G J, a + r < Z„ < a + s] 

j<k 

+ P[max|Cn -Cn-il > e], 

where Z* = £„ + Cn! and min :) <fc(Z J 1 i — Z^_^) > r — e iff W„ G 5, where 

B = {w : x-j + i H h x a + C(w ) - C(w_j-+i) > r - e, for j = 1, • • • , k}. 

So, 

P{r<Ra<s,Z ta £l,(t a € J] 

OO 

< ^ P i W n G P, Cn G /, Cn G J, a + r < Z n < a + s] + o(l), 

n=l 

and 

Umsup P[r <R a < s,C* a G I,(t a G J] < (^^)P[W G P,Co G /]£{./}, 

a — >oo 

by Theorem [TJ Letting e — > and k — > oo, 

limsup P[r < P a < s,Ct a G J,Ct. G J] 

SO 

< ( — )P[.mf (Zo 1 - Zi 3 ) > r,Co G 7]L{J} 



(22) Q ^°° 
v 1 ,s — r 



M J<- 

By partitioning an interval (r, s] into subintervals r = ?"o < 7*i < • • • < r m = s, 
applying (|22p to each subinterval, and letting the partition become infinitely fine, 
it yields 

1 r 

lim P[r <R a <s, Ct a G I, C ta G J = — / P inf Z* > u, Co < V du X L(z), 

a-+oo |U J r j<-l 



from which the theorem follows by using (|2ip . □ 
Thus, the asymptotic distribution functions of R a and Ct a are 

- f P\ inf Z* > slds and - f \ inf Z*]+dP. 
Denote the means of these distributions by p and v. 
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Theorem 4. If p x > 2/(1 — q), then 

(23) lim .E|— --|=0. 

a^oo a /i 

If also p y > (2 — q)/q and ^ holds with k = 1, 

, , a + o — v — A 

(24) E(ta) = +o(l) 

M 

as a — > oo, where A = Ai + • • • + Ad, and Ai, . . . , A<j are as in l[8]). 
Proof. From Lemma [T] i a /a — > p l//i as a — > oo, and 

r. OO 

(25) / tdP < MP[t > M] + ^2 P[t> n] -> 0. 

Relation ([2"3"| follows. 

For pg|). first observe that P(A') = o(l/a), by Lemmas □ and H From Wald's 
Lemmas and ([23]), #(5 t J = ^J5(t ) and E[(S ta ~ ^a) 2 ] = cr 2 E(t a ) = 0(a). So, by 

M / t«dP = / + / (S ta - flQdP, 

J A J A J A' 

and 



| / (5 ta - flt a )dP\ < y/P{A')y/E{St a -flt a ) 2 = o(l). 

So, using Theorem [3] and Corollaries [1] and [21 

H / i„dP - aP(A) + [ (R a -£ ta -( ta )dP + o(l)=a + p-v-\ + o(l) 

J A J A 



Finally, 



/ t a dP <MP{A')+ [ t a dP^0, 
J A' Jt„>M 



I A 1 Jt a >M 

so that E(t a ) = J A t a dP + o(l). Relation ([24|) follows. □ 

Remark. In fact, ([23)1 can be established under the weaker condition p x > (2 — 
q)/(l — q), at the expense of complicating the proof in fairly routine ways. 

4. Examples 

Consider the exponential model described in the Introduction. In this model, many 
examples are of the form 

(26) Z n = ng(^,^), 

n n 

where g is a smooth function. This can be written in the form |T]) with 

1 1 " 1 

S n = ng(l, -) + g 01 {l, -)^2(L k - -), 

" fe=i 
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= -[510(1, -) ^ l{L n - k > ^Vn-j} +50l(l, -)^ (L„_fe — 

and 

Cn =n 5 (^,^)-[5„+^]. 
n n 

where 77/c = — Tk-\ and <?jj denote the partial derivatives of g. The latter term is 
discussed in more detail below. 

Example 1 (Fixed width confidence intervals). Consider the problem of setting 
a confidence interval with fixed width 2ft, > for 9 in the Exponential model. 
Asymptotic considerations lead to a stopping time of the form t a with 

rr>* 

and a — c 2 /h 2 , where c is a normal percentile, as in This is of the form (f2T))) 
with g(x,y) = y 2 /x 2 . □ 

Example 2 (Repeated significance tests). The logdikelihood ratio test statistic for 
testing 9 = 1 is 

Z n =K n log(^)+(T;-K n ), 

n 

which is of the form (|26p with g(x 7 y) = x\og(x/y) + (y — x). □ 
Returning to Equation (|26p . Cn may be written 

Cn = Cl,n + (,2,n + (3,711 

where 

Ci.n = ~n[<?02(l ,I)(^-i)» + 2 fln (l + -I) 2 ]. 

2 9 n 9 9 n 9 n 9 n 



k 

''ln-jj + i 



&,n = 510(1, X! > X! »7«-J'} + ffOiC 1 ' X! ( L "- fc ~ X] 7 ?" 

fc=n J— /c— n j— 

and 

C3,n = ^n — (Sn + £,n + Cl,n + C2,n), 

which may be bounded by a constant multiple of n[\T*/n — l/9\ 3 + \K n /n — 1| ] 
if \9T*/n - 1 1 < 1/2 and if n /n > 1/2. Further, letting Yfc = i fc -1/9 and T„ = 

_ ±_ _ ^_n_ _ £n 

n 9 n n 

where 

n 

£n = /J^n-fc+1 - (r„ - T„_fc)] + . 
fc=l 

So, £„ may be written in the form (j4| with 

C = ^5o 2 (l, 1)T„ 2 

and 

Cn = Cl'n + C2,n + C3,n 
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with 



— + 2 ffO2(1 '0 ) ^T 



"flu 



n n 



n 



— 1 



There are several terms to be considered in £^', but none causes much difficulty. 

'3=0 



Some selected details: First P[C2,n 7^ 0] < P[Xfc > X^=o % f° r some ^ > "■], which 
is decays exponentially as n — > oo. For P|£°| p < oo (independent of n) for all 
p > 0, and E\T n \ 2 P = 0(n p ) for all p > 0. So, P|C"„| P = 0(n"3P) for all p. Finally, 
let P„ be the event that \9T n /n - 1\ < 1/2 and if„ > ra/2. Then P(PvJ decays 
geometrically, and if B n occurs, then is bounded by a constant multiple of 



IT* - -I 



\K„ 



The conditions ^ and (J7J) can be verified using these facts. 



5. Appendix 

The Baum-Katz Inequalities are used in the following form: If A > and n > I > 1 
are positive integers, then 

- A /■U 2 na 2 \ l 
P[max \S k - > A] < nP(-) + {-^-) , 

where P(A) = P[|Xi| > A]. See (pp. 373-375). 

First consider Ao. For any fixed n, P[Z n > a] — > as a — > 00. So, it suffices to 
consider $^ n _/ P[^n > a] for any t. If n < m, then 

P[Z„ > a] < P[S„ - n M > + P[£„ + C„ > ^^]. 

By 0, C = sup„> P|£„ + C«| 2 < 00. So, 

m m . „ 



n— no 



as a -> 00. Next, let 2 > 1/(1 — 2g). Then 



E„ r „ a — nil. \ -.a — nu. (\btao\ l 

P[S n - n» > —^] < ^ ^(-2^) + 

n— no n— 1 



which approaches as a — > 00, after some simple analysis. The analysis of Ai is 
similar, using 

P[^n < a + ia 1 ^] < P[|S„ - H > !! ^] + P[|£n + Cn| > 

<nF( ¥_fl )+ rl6^ 2 MCT 2 ,£ + 16C* 
— 4 (n/i — o) 2 (n/i — a) 2 

and P[i a > n] < P[Z n < a] for n > M. Relation ^ follows. 
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For O, 




32l 2 Mo- 2 ,i 32aC 
a 2(i-«) ' + a 2(l-g) ' 



which approaches as a — ► oo if p K > 2/(1 — q) and £ > 1/(1 — 2q). That leaves 



P[< a < m]. By ©, C" = sup„> no E\£ n + C„ | 7 < oo for some 7 > (2-q)/(l-q). So, 
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P[t a < m] < P[max \Sk — kfi\ > —a 



1+ E *+Cn|> gCo-n/*)] 




which is o(l /a) if I > 1/(1 - 2q). 



□ 



